Skip to content

fix(compose): forward batch tokens and strict Jev knobs to laya-serve - #1054

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
NAVEENPRASAATH23:fix/compose-batch-tokens-passthrough
Oct 10, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
NAVEENPRASAATH23:fix/compose-batch-tokens-passthrough

Conversation

@NAVEENPRASAATH23

@NAVEENPRASAATH23 NAVEENPRASAATH23 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

What

  1. Forwards LAYA_MAX_BATCH_TOKENS and LAYA_JEV_STRICT from host environment variables through compose.http.yaml to the laya-serve container.
  2. Documents both in docs/docker.md's Server configuration table and laya/serve.py module docstring.
  3. Pins their passthrough and defaults in tests/test_compose_env.py.
  4. Includes the TypeScript compilation error fix for laya-ts/src/hooks.ts and laya-ts/src/agent.ts to keep the TypeScript CI suite green.

Why

laya.serve reads two runtime knobs for batch inference and client compatibility:

  • LAYA_MAX_BATCH_TOKENS: budgets tokens per forward pass for POST /v1/systemone/batch (DEFAULT_MAX_BATCH_TOKENS = 131072), chunking requests that exceed it to protect workers against OOMs.
  • LAYA_JEV_STRICT: projects responses onto the strict Jev wire contract (stripping extra metadata fields that cause validation failures on strict Jev clients).

Both are documented in docs/http-api.md and wired into nix/laya-serve.nix, but were omitted from compose.http.yaml and docs/docker.md. When an operator configured them for a Docker Compose deployment, Compose silently dropped the variables because they were not declared under services.laya-serve.environment.

This achieves 100% reachability and parity across all server runtime configuration dials in Compose.

How it was verified

Executed the test suites locally with zero errors:

python tests/test_compose_env.py
# 258 passed, 0 failed

python tests/test_env_docs.py
# 27 passed, 0 failed

python tests/test_packaging.py
# 138 passed, 0 failed

ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
# All checks passed!

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

@NandhaKishorM
NandhaKishorM merged commit e1b505b into NandhaKishorM:main Oct 10, 2026
40 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged.

Same shape as #1053 and the same real gap: LAYA_MAX_BATCH_TOKENS and LAYA_JEV_STRICT were documented and read by the runtime, and compose did not forward either, so the service that needs them was the one that never named them.

LAYA_JEV_STRICT is the one with the sharper consequence, because it decides the response shape. An operator who sets it expecting strict Jev projection and silently gets the default payload has clients parsing fields that were supposed to be absent.

Merge note: this and #1053 both extend the same name tuple in tests/test_compose_env.py, in two places, so they conflicted. Both sets are kept, all four names, and the paired value tuple is now a single _VALS constant so the names and the values cannot drift apart the next time someone adds a knob. tests/test_compose_env.py passes 284 of 284 on the merged result.

NandhaKishorM added a commit that referenced this pull request Oct 10, 2026
14 pull requests from 8 contributors.

* the TypeScript tree parses again: an unterminated template literal shipped in
  0.4.1 and stopped 22 of 34 test files loading (#1056)
* x_jev_confidence on /v1/systemone, computed locally (#1066, #302)
* a narrowed shortlist drops option_order with a warning instead of failing (#1063)
* registering a name one edit from a built-in warns (#1055, #919)
* compose forwards four knobs the runtime already read (#1053, #1054)
* Java Router.predictBatch and Shortlist.predictTournament (#1060, #1061)
* teacher-agreement evals, with the self-agreement ceiling (#1071, #919)

Security: no malicious content found; the audit is recorded locally.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants